Papers with query dependent context modeling
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models excel in generating coherent and contextually rich outputs, but their capacity to handle long-form contexts is limited by fixed-length position embeddings. |
| Approach: | They propose a method that enables the efficient processing long-form sequences beyond the model’s length limit through recurrent compression without retraining the entire model. |
| Outcome: | The proposed method significantly improves LLM’s ability to manage extended contexts, making it well-suited for tasks that require both comprehensive context understanding and query relevance. |